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abstract 


Asymptotic  procedures  are  given  for  testing  certain  hypotheses  con¬ 
cerning  eigenvectors  and  for  constructing  confidence  regions  for  eigen¬ 
vectors.  These  asymptotic  procedures  are  derived  under  fairly  general 
conditions  on  the  estimates  of  the  matrix  whose  eigenvectors  are  of 
interest.  Applications  of  the  general  results  to  principal  components 
analysis  and  canonical  variate  analysis  are  given. 
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1.  INTRODUCTION  AND  SUMMARY 


LeC  M  be  a  (p  x  p)  matrix  which  is  symmetric  in  the  metric  of  the 

positive  definite  symmetric  matrix  T,  i.e.  TM  is  symmetric.  Let  the 

eigenvalues  of  M  be  represented  by  X^  >  X^  ...  >  X^.  Also,  let  be 

a  sequence  of  estimates  of  M  such  that  a  (M  -  M)  converges  in  distribution 

n  n 

to  a  multivariate  normal  distribution,  where  a  is  an  increasing  sequence 

n 

of  real  numbers,  and  let  A  be  an  (r  x  p)  matrix  with  rank(A)  =  r. 

In  this  paper,  under  the  assumption  that  X.  ,  ^  X .  and  X.  ,  f  X 

l-l  i  i+m-1 


l+m 


the  following  null  hypothesis  is  considered. 


(1.1)  Hq:  the  columns  of  A  lie  in  the  subspace  generated  by  the  set  of 

eigenvectors  of  M  associated  with  the  roots  X .,  X X .  , . 

l  l+l  i+m-1 


The  assumption  on  the  eigenvalues  is  to  be  interpreted  as  X.  f  X. 

i+m-1  l+m 

when  i  =  1,  and  X^_^  f  X^  when  i+m-1  =  p. 

Under  fairly  general  condition  on  M  ,  a  consistent  asymptotic  chi- 

n 

square  test  of  is  given.  This  test  is  based  upon  the  asymptotic 

normality  of  the  "orthogonal"  projection  of  the  columns  of  A  onto  the 

subspace  generated  by  the  eigenvectors  of  M  associated  with  the  ith  to 

n 

(i+m-l)th  roues  of  M  . 

n 

An  asymptotic  confidence  region  for  the  subspace  generated  by  the 

eigenvectors  of  M  assoicated  with  the  roots  X.,X.  X.  .is 

li+l  i+m-1 

then  given.  This  confidence  region  is  based  upon  the  asymptotic  chi- 
square  test  of  for  the  special  case  when  r  =  m.  Furthermore,  an 
asymptotic  chi-square  test  is  given  for  the  hypothesis  that  the  subspace 
generated  by  the  columns  of  K,  a  (k  x  p)  matrix  with  rank(K)  =  k  5  m,  con¬ 
tains  the  subspace  generated  by  the  eigenvectors  of  M  associated  with  the  roots 
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A.,  A.^...,  ^j_+ra-i  •  This  test  is  constructed  by  relating  this  hypothesis 
to  a  hypothesis  of  the  form  given  in  (1.1). 

Anderson  (1963)  gives  an  asymptotic  chi-square  test  of  Hq  for  the 
special  case  of  m  =  1  and  when  is  the  sample  covariance  matrix  from 
a  multivariate  normal  sample  with  population  covariance  matrix  M.  This 
paper  is  thus  a  generalization  of  Anderson's  results. 

James  (1977)  gives  an  exact  test  for  a  hypothesis  similar  to  (1.1) 
when  is  the  sample  covariance  matrix  from  a  multivariate  normal 
sample  with  population  covariance  matrix  M.  James  considers  the 
hypothesis  that  the  columns  of  A  generate  an  invariant  subspace  of  M. 

His  hypothesis  does  not  state  with  which  eigenvalues  of  M  the  invariant 
space  is  associated.  The  approach  used  by  James  uses  special  properties 
of  the  ample  covariance  matrix  from  a  normal  sample  and  does  not 
readily  generalize  to  other  matrices. 

For  other  related  works  on  the  distributional  and  inferential  theory 
for  eigenvectors,  the  reader  is  refered  to  Anderson  (1951),  Mallows  (1961), 
Chambers  (1967),  Izenman  (1976)  and  Suguira  (1976). 

Applications  of  the  general  results  in  this  paper  are  illustrated 
through  the  following  two  examples:  the  principal  component  vectors  for 
the  covariance  matrix  of  a  multivariate  normal  distribution,  and  the 
canonical  vectors  associated  with  two  random  vectors  which  jointly  have 
a  multivariate  normal  distribution. 
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2.  PRELIMINARIES 


In  this  section,  let  S  be  a  (q  *  q)  real  matrix  which  is  symmetric  in 
Che  metric  o£  a  real  positive  definite  symmetric  matrix  T.  In  order  to 
establish  notation  and  vocabulary,  the  eigenvalue  problem  for  S  is  briefly  re¬ 
viewed  below.  A  more  detailed  review  can  be  found  in  Kato  (1966)  or 
Nerring  (1970). 

If  Sx  =  Xx  for  some  x  f  0,  then  X..is  an  eigenvalue  of  S  and  x  is  an  eigenvector 

of  S  associated  with  X.  All  eigenvalues  of  S  are  real.  The  spectral  set 
of  S,  denoted  s,  is  the  set  of  all  eigenvalues  of  S-  Eigenvalues  of 
"symmetric"  matrices  have  the  following  important  continuity  property. 

LEMMA  2.1.  If  the  (q  x  q)  matrix  is  symmetric  in  the  metric  of 

T  ,  with  eigenvalues  X  (S  )  2  X  (S  )  2  ...  2  X  (S  ),  and  S  •*  S  as 

tC  1  K  Z  K  ^  K  K 

k  -*■  ®,  then  X .  ( S,  )  -*■  X.(S)  as  k  -*■  00 . 

J  k  j 

The  eigenspace  of  S  associated  with  X  is  V(X)  =  {x  e  (  Sx  =  Xx} , 
q 

where  R  is  the  set  of  all  q-dimensional  real  vectors.  The  dimension  of 
V(X)  is  the  multiplicity  of  X,  say  m(X).  If  X  and  y  are  two  distinct 
eigenvalues  of  M,  then  V(X)  and  V(y)  are  orthogonal  subspaces  in  the  metric 

of  T.  That  is,  if  x  e  V(X)  and  y  e  V(y),  then  x'Ty  =  0. 

Since  S  is  symmetric  in  the  metric  of  T,  we  have  the  decomposition, 

=  £^SV(X) The  eigenprojection  of  S  associated  with  X,  denoted  P(X), 
is  the  projection  operator  onto  V(X)  with  respect  to  this  decomposition 
of  R^.  If  V  is  any  subset  of  the  spectral  set  s,  then  the  total  eigen¬ 
projection  for  S  associated  with  the  eigenvalues  in  v  is  defined  to  be 

P(X),  For  any  set  of  vectors  {x.}  in  V(X)  such  that 
A6l?  J 
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ijTik  =  w^ere  denotes  the  Kronecker  delta,  P(X)  has  the 

representation  P(A)  =  Thus  P(A)  is  symmetric  in  the  metric 

of  T. 


The  spectral  decomposition  of  S  is  S  =  E^^Ap(A).  If  all  the  eigen¬ 
values  of  S  are  non-negative,  then  the  square  root  of  S  is  to  be  defined 

as  =  £■,  A^P(X). 

Acs 

A  generalized  inverse  of  S  is  any  S  such  that  SS  S  =  S.  The  Moore- 
Penrcse  generalized  inverse  of  S,  denoted  by  S+,  can  be  represented  by 

s  =  hfs.xto  k  lp(X)- 

In  working  with  random  matrices,  it  is  necessary  to  introduce  the 
following  notation.  If  B  is  a  (b  x  t)  matrix,  then  vec(B)  is  the 
transformation  of  B  into  a  bt-dimensional  vector  in  the  following  fashion. 
Let  B  =  Cbx  b2  ...  b^],  where  b^  is  the  jth  column  of  B,  then 


(2.1) 


vec(B)  = 


If  B  is  a  (b  x  c)  matrix  and  C  is  a  (c  *  u)  matrix,  then  the 
Kronecker  product  of  B  and  C  is  the  (be  *  tu)  partitioned  matrix 
B®C  =  Cbj^Cl,  j  =  1,  2,  ...,  b  and  k  =  1,  2,  ...»  t  with  j  varying 
over  rows  of  matrices  and  k  varying  over  columns  of  matrices. 

An  important  property  relating  the  "vec"  transformation  and  the 
Kronecker  product  is 


(2.2) 


vec (BCD)  =  (Df  ®  B)vec(C), 


where  the  dimensions  of  the  matrices  B,  C,  and  D  are  such  that  the  multi¬ 
plications  are  properly  defined.  Other  properties  of  the  "vec"  transfor¬ 
mation  and  the  Kronecker  product  can  be  found  in  Neudecker  (1968,  1969). 

The  commutation  matrix  or  permuted  identity  matrix  is  the  (ab  *  ab) 
a  b  • 

matrix  1/  ..  =  I.  , E..  ©  E..,  where  E. .  is  at  (a  »  b)  matrix  with 

(a.b)  i=l  j=l  ij  ij ’  ij 

a  one  in  the  (i,j)  position  and  zeroes  elsewhere.  The  commutation  matrix 
has  been  extensively  investigated  recently  by  Magnus  and  Neudecker  (1979). 
Two  important  properties  of  the  commutation  matrix  are 

(2*3)  I(a  ^  vec(B)  =  vec(B'),  and 

(2.4)  I  (C  ®  D)  =  (D  ©  C)I,  .x, 

la,b;  1c,  d ; 

where  B  is  (b  *  a),  C  is  (b  *  d),  and  D  is  (a  *  c). 

If  Y  is  a  random  vector,  let  var(Y)  represent  the  covarinace 
matrix  of  Y.  If  B  is  a  random  matrix,  then  for  convenience,  var[vec(B)] 
is  to  be  written  as  var(B). 

3.  ASSUMPTIONS 

In  order  to  form  an  asymptotic  test  for  (1.1),  a  sequence  of  estimators 
Mn  ^or  M  are  needed  which  satisfies  the  following  assumptions. 

ASSUMPTION  3.1. 

(i)  M  is  symmetric  in  the  metric  of  T  ,  a  positive  definite  sym- 

u  q  * 

metric  matrix,  with  r  T  in  probability. 

(ii)  an(MQ“M)  "*■  N  in  distribution,  where  a^  is  an  increasing  se¬ 
quence  of  positive  numbers  such  chat  a^  -*■  <=  as  n  ®,  and  N  is  a  multi¬ 
variate  normal  matrix  with  zero  mean  and  var(N)  =  I. 

(iii)  For  B  which  is  (p  *  p) ,  Jvec(TB)  =  0  implies  M(B  +  B  ')  =  0. 


Ic  is  also  necessary  Co  have  a  sequence  of  estimators  for  I  which 
satisfies  the  following  properties. 

ASSLTMPTION  3.2 

(i)  is  symmetric  and  positive  semi-definite. 

(ii)  X  I  in  probability. 

n 

(iii)  Let  SI  =  {I  vec(F  B)  =  0  implies  M  (B  +  B1)  =  0}  then 

n  n  n  n 

Prob(Sl  )  -*■  1 . 
n 


Finally,  it  is  to  be  understood  that  the  asymptotic  procedures  given 
in  this  paper  are  only  defined  on  the  set 

A  A 


(3.1) 


C  =  {X.  5*  X.  and  X.  ^  X .  }  , 

n  i-l  i  L+m-1  l+m  ’ 


where  X  >  X  >  . . .  >  X  are  the  eigenvalues  of  M  .  It  is  irrelevant 
1  +  P  n 

to  the  asymptotic  properties  of  the  procedures  what  action  is  taken  on 

the  complement  of  C  ,  since  by  the  continuity  of  the  eigenvalues  of 
n 

"symmetric"  matrices,  that  is  Lemma  2.1,  ProbCC  ]  +  1 . 

n 


4.  ASYMPTOTIC  DISTRIBUTION  OF  THE  EIGENPROJECTION 

A  ,  A  A 

Let  w  —  {X.,  X  ,  ...,  X  }  and  let  w  = '  (X  ,  . . . ,  X  }. 

l  l+l  i+m-1  l+l  i+m-1 

Also,  for  X  an  eigenvalue  of  M,  let  P^  represent  the  eigenprojection  of 
M  associated  with  X,  and  let  =  (M  -  Xl)+.  For  X  an  eigenvalue  of 

A 

M^,  let  P^  represent  the  eigenprojection  of  associated  with  X,  and 

A  +  A  A 

let  Q,  *  (M  -  Al)  .  For  convenience,  define  P  =  P,  and  P  =  E,  aP.  # 
An  o  A£w  A  o  Aew  A 

Pq  represents  the  total  eigenprojection  of  M  associated  with  the  eigenvalues 
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A 

of  M  in  w,  and  P  represents  the  total  eigen-projection  Qf  jh  associated 
°  '  n 

with  the  eigenvalues  of  M  in  w. 

n 

The  null  hypotheses  (1.1)  can  thus  be  rephrased  as 

(4.1)  H  :  P  A  =  A 

o  o 

where  A  is  (p  *  r)  with  rank(A)  =  r. 

A 

A  natural  statistic  to  consider  in  testing  H  is  (P  A  -  A) .  In 

o  o 

obtaining  the  asymptotic  distribution  of  this  statistic,  the  Taylor 

A 

series  expansion  of  Pq  about  P^  is  to  be  used.  This  expansion  is  given 
in  the  following  lemma.  The  lemma  is  a  simplified  version  of  more 
general  results  given  in  Chapter  2  of  Kato  (1966).  A  proof  of  this 
simplified  version  can  be  found  in  Appendix  B  of  the  author's  dissertation. 


(4.4)  I  (A)  =  A'tA  ,  where 

o  o  o 

A  =  -  [E.  P  Q  (I-P')Q'](A  0  I)  =  E.  E  .  (A-p)~^P  A  0  P '  . 
o  Aew  A  o  A  Aew  uiw  A  U 

A 

In  Che  next  section,  the  asymptotic  normality  of  a^P^A  -  A)  is  to  be  used 
in  forming  an  asymptotic  chi-square  test.  Before  doing  so,  Che  rank,  of 
I  (A)  is  needed. 


THEOREM  4.2.  If  the  columns  of  A  are  in  the  range  of  Pq,  then 

rankCI  (A)3  *  (p-m)r. 
o 

PROOF.  The  proof  consists  of  determining  the  null  space  of  Iq(A) . 

For  G  which  is  (p  *  r),  Cvec(FC) J ' Xq(A)  vec(FG)  =  Lvec(FB) ] ' Ivec(FB) , 

where  B  =  E^£wE^w(A-p)  ^P^Ga’P-^.  So  by  Assumption  2.1.iii,  t^CAjvecCFG)  =  0 

implies  M(B  +  B')  =  0,  which  implies  P^G  =  0  for  p  •{  w. 

The  last  implication  is  justified  by  the  following  contrapositive 
argument.  Suppose  p  i  w  and  P^G  f  0.  Case  I:  0  <  w. 

M(B  +  B')P  =  (E.  A(A-p)  ^P  )AG'P  f  0  since  rank[(E,  A(A-p)  ^p.  )A  =  r. 
p  Aew  A  p  Aew  A 

Case  II:  0  e  w.  P  M(B  +  BT)  =  pP  GA’(2,  (A-p)_1P, )  +  0 

P  11  ACW  K 

The  converse,  that  is  P|jG  =  0  for  all  P  4  w  implies  Jo(A)vec(rc)  =  0. 

is  obviously  true.  Thus,  the  null  space  of  J  (A)  is  n  =  {vec(i'G)  i  Cl~P  )G  =  0}. 

o  o 

Thus,  dimension(r|)  =  mr,  and  so  rankLJQCA)j  =  (p-m)r. 


5.  AN  ASYMPTOTIC  CHI-SQUARE  TEST 

In  this  section,  an  asymptotic  chi-square  test  of  H  is  given  which 

A 

is  based  upon  the  asymptotic  normality  of  a  (P  A  -  A).  Due  to  the 

n  o 

singularity  of  I  (A),  the  chi-square  test  and  its  properties  are  not 
o 

straightforward.  So,  for  clarity,  most  proofs  for  this  section  are 
given  in  the  appendix. 
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THEOREM  5.1.  Let 


t0(A)  -  (A*  ©  I )[E  A  p*  ©  Q  (i-p  )]J  [E  A  P  ©  (I-P')Q’](A0  I), 
v  ACW  A  A  On  A€W  A  O  A 

A  A 

and  let  Cl  (A)]  represent  any  generalized  inverses  of  X  (A) . 
u  o 

Also,  define  T  (A)  =  a2{vec[(P  -  I)A]}'CJ  (A)]  vec[(P  -  I)A]. 

n  n  o  o  o 

2 

Then,  under  H  ,  T  (A)  -*•  X  /•  \  in  distribution, 

o  n  r(p-ra) 


PROOF.  See  the  appendix. 


Theorem  5  states  that  the  limiting  distribution  of  T  (A)  under  H 

n  o 

A 

does  not  depend  upon  the  choice  of  generalized  inverses  for  I  (A).  The 

o 

next  theorem  states  that  asymptotically  the  value  of  T  (A)  does  not  depend 

A 

upon  the  choice  of  the  generalized  inverses  for  l  (A). 

o 

THEOREM  5.2.  Let  T  (A)  be  defined  as  in  Theorem  5.1. 
n 

A 

(i)  On  the  set  {rank(P  A)  =  r},  T  (A)  is  invariant  under  different 

o  n 

.  A 

choices  of  a  generalized  inverse  for  l  (A). 

o 

(ii)  Whether  or  not  H  is  true,  Prob[rank(P  A)  =  r]  -*•  1. 

o  o 


PROOF.  See  the  appendix. 


So,  on  the  set  {rank(P  A)  =  r} ,  T  (A)  is  unique  and  the  Moore- 

o  n 

A 

Penrose  inverse  for  $  (A)  can  thus  be  used  on  this  set.  In  addition,  if 

o 

M  is  symmetric,  Chen  T  (A)  has  the  representation 
a  n 


(5.1) 


Tn(A)  =  a2Cvec(A)],CIo(A)]+vec(A) 
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A 

on  the  set  {rank(PQA)  =  r}.  This  statement  is  justified  by  noting  that 
for  a  symmetric  matrix  B,  Bx  =  0  if  and  only  if  B  x  =  0,  and  it  is  easy 

A  A 

to  verify  that  1  (A)vec(P  A)  =  0. 

a  o  ~ 

Now,  application  of  Theorems  5.1  and  5.2  gives  the  following  asymptotic 

a  level  test  of  H  • 
o 

(5.2)  Reject  H  if 

o 

A 

(i)  rank(P  A)  <  r.  or 

o 

A  2 

(ii)  rankCP^A)  =  r  and  T^(A)  >  ^  ^  (a) , 

2  v  2 

where  X^CoO  is  the  (L-a)  percentile  of  a  X  distribution. 

By  Theorem  5-2. ii,  it  is  irrelevant  to  the  asymptotic  properties  of 

A 

a  test  of  H  what  action  is  taken  on  the  set  {rank(P  A)  <  r} .  However, 
o  o 

rejecting  for  this  case  enables  the  rejection  region  to  be  "continuous" 

A 

in  the  sense  that  for  any  sequence  {A^}  such  that  rank(PQA^)  =  r  and  A^  -*•  A, 

A 

where  rank(P  A)  <  r,  then 
o 

(5.3)  T  (A,  )  -*•  °°,  as  k  +  00 . 

n  k 

The  proof  of  property  (5.3)  is  given  in  the  appendix.  Property  (5.3)  is 
important  when  using  the  test  defined  by  (5.2)  for  constructing  confidence 
regions  for  the  range  of  P  .  This  is  done  in  the  next  section. 

Although  the  test  of  defined  by  (5.2)  was  intuitively  motivated, 
it  does  have  the  following  important  properties. 

THEOREM  5.3 

(i)  (5.2)  is  a  consistent  test  of  H  .  That  is,  if  H  is  not  true, 

o  o 

then  ProbLre jecting  H  ]  -*■  1. 

(ii)  (5.2)  is  invariant  under  post-multiplication  of  A  by  a  non¬ 
singular  matrix. 

PROOF.  See  the  appendix. 
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Theorem  5.3.ii  is  important  since  the  hypothesis  Hq  is  invariant  under 

post-multiplication  of  A  by  a  non-singular  matrix.  Thus,  the  test  given 

by  (5.2)  tests  whether  the  space  spanned  by  the  columns  of  A  is  a 

subspace  of  the  range  of  P  . 

o 

REMARK  1.  If  the  assumption  A.  ,  f  X,  or  the  assumption  X.  f  X . 

r  1.-1  l  i+m-1  i+m 

is  false,  then  the  asymptotic  chi-square  test  given  by  (5.2)  is  not 

generally  valid.  If  the  assumptions  on  the  eigenvalues  are  true,  then  by  Lemma 

4.1,  the  "sample"  size  n  necessary  to  insure  that  the  asymptotic  chi- 

square  test  is  a  "good"  approximation  is  in  general  inversely  related  to 

the  quantity  min(A  -  X. -  X.^) .  In  addition,  if  is  "close" 

to  A^,  one  may  not  wish  to  study  the  eigenspace  associated  with  A^_^ 

separately  from  the  eigenspace  associated  with  A^.  So,  in  practice, 

before  determining  which  eigenspaces  are  of  interest,  a  study  of  the 

eigenvalues  would  be  desirable. 

REMARK  2.  Let  v  =  (A^,  j  c  i},  where  I  is  some  index  set.  Under 
the  assumption  A^  ^  for  all  j  £  I  and  k  4.  I,  consider  the  hypothesis 

(5.4)  the  columns  of  A  lie  in  the  subspace  generated  by  the 

eigenvectors  of  M  associated  with  {X.,  j  e  i} , 

J 

where  A  is  (p  *  r)  with  rank(A)  =  r  £  m  =■  rankd^P^) .  This  hypothesis 
can  be  tested  by  using  the  test  given  by  (5.2)  provided  w  is  replaced  by 

A  A  ^ 

v  and  w  is  replaced  by  v  =  {X.,  j  e  i). 

REMARK  3.  Under  the  assumption  X.  j4  A.  and  A.  j4  A .  ,  consider 

i~l  t  i+m-1  i+ra 

the  hypothesis 

(5.5)  H  :  the  eigenvectors  of  M  associated  with  the  roots  A., A  A 

0  i.  l+1  ’  i+m-1 

lie  in  the  subspace  generated  by  the  columns  of  A, 
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where  A  is  (p  x  r)  with  rank(A)  =  r  >  m.  This  hypothesis  can  be  tested 
by  using  the  following  approach.  Let  3  be  [p  *  (p-r)]  with  rank(3)  =  p-r 
and  such  that  A'B  =  0.  The  hypothesis  (5.5)  can  then  be  rephrased  as 


(5.6) 


H^:  the  columns  of  B  lie  in  the  subspace  generated  by  the 

eigenvectors  of  M'  associated  with  the  eigenvalues 


W 


«.|X.  . |X.  , 

l-l  l+m 


...X, 


Note  that  if  M  is  symmetric  in  the  metric  of  T,  then  M '  is  symmetric  in 

the  metric  of  T  \  It  is  easy  to  verify  that  if  the  conditions  on  M,  T, 

M  ,  F  ,  X  and  X  given  by  Assumptions  3.1  and  3.2  are  satisfied,  then 

n  n  n 

the  conditions  are  satisfied  when  M,  T,  M  ,  T  ,  X  and  X  are  replaced  by 

n  n  n 

m',  T  \  M ' ,  T  ^ ,  var(N')  =  1,.  ,  t  I,  ^  and  I,  .  X  I,  ..  respec- 

n’  n  (p.p)  (p,p)  (p.p)  n  (p,p)  r 

tively.  So,  by  remark  2,  the  results  of  this  section  apply  to 
testing  the  hypothesis  (5.6). 

Note  that  if  r  =  m,  then  the  hypothesis  (1.1)  and  (5.5)  are  equivalent. 
For  this  case,  the  test  given  by  (5-2)  when  applied  to  the  hypothesis 
(1.1)  is  the  same  as  the  test  for  (5.5)  suggested  in  this  remark. 


6.  ASYMPTOTIC  CONFIDENCE  REGIONS 

For  A  which  is  (p  *  m) ,  let  L(A)  represent  the  space  spanned  by  the 
columns  of  A.  That  is, 

L(A)  =  {v  c  RP  I  v  =  Aw  for  some  w  e  Rm} . 

The  test  of  hypothesis  (1.1),  when  r  =  m,  given  by  (5.2)  yields  the 

following  asymptotic  (l~a)  confidence  region  for  the  range  of  P  , 

o 

(6.1)  (L(A)  I  A  is  (p  *  m),  rank(A)  =  ra,  and  T  (A)  <  xlr  \(a)K 

n  mu  p~m; 
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One  undesirable  aspect  of  tms  confidence  region  is  that  T  (A) 

n 

A 

involves  a  generalized  inverse  of  I^CA),  which  must  be  recalculated  for 
each  A.  However,  this  problem  can  be  alleviated  and  the  confidence 
region  can  be  given  a  simplier  representation. 


To  make  the  simplification,  let 


(6.2) 


X  =  [x.  £..  ...  £.  .], 

n  ~i  ~i+l  ~i+m-l 


■  A  ,  ,  A  A  A  A  A 

where  ix.)  is  defined  such  that  M  x.  =  A.x.,  and  x.'T  x  --6...  By 
~J  n~j  j~j  ~j  n~k  ij 

A 

noting  that  P  *  X  X  i  ,  it  can  then  be  easily  verified  that 
o  n  n  n  1 

i  (a)  =  (a't  x  ©  or  (x  Kx'r  A  ©  I). 

o  n  n  o  n  n  n 

So,  by  Theorem  5.3. ii,  if  rank(P  A)  =  m,  then 

o 

Tn(A)  =  TnCA(XnrnA)  1]  =  *^vecCA(X'  rA)'1  -  Xj}  ’r/X^+vecCAUT^)"1  -  X  J . 
Thus,  (6.1)  can  be  rewritten  as 


(6.3)  {L(A)  I  XT  A  =  I  and  a2[vec(A-X  )]'I  (X  )+vec(A-X  )  <  y' .  .(a)}. 

n  n  n  non  n  m(p-tn) 


For  the  special  case  m  =  1,  (6.3)  reduces  to 


(6.4)  (ca  I  x!T  a  *  1,  and  a2(a-x. )  TA+(a-x. )  <  y2  .(a)}, 

~  i  n~  n  ~  ~i  n - i  p-1 

where  A  =  [x!  ©  (M  -A.I)+3r  Cx.  ©  (M’-A.I)+]. 
n  ~i  n  i  n  ~i  n  l 


If  and  M  are  symmetric,  (6.3)  and  (6.4)  respectively  reduce  to 

(6.5)  {L(A)  |  X'A  =  I  and  a2[vec(A)]'r  (X  )+[vec(A)]  <  ^  (ct) } , 

n  n  on  m(p-m) 


(6.6)  {ca  |  x!a  ■  1,  and  a2a'A+a  <  X2  ,(ci)}, 

~  ~i  n~  n~  p-1 

where  A  =  Cx!  ©  (M  —A . I ) + ] J  Lx.  ©  (m  -A.I)+]. 
n  ~i  n  ~*i  n  ~i  n  i 


7.  EXAMPLES 


I)  PRINCICAL  COMPONENTS  ANALYSIS.  One  of  the  most  common  uses 

of  eigenvectors  in  statistics  is  in  the  principal  components  analysis 

of  a  covariance  matrix  for  a  multivariate  normal  distribution.  For 

this  case,  M  is  taken  to  be  the  sample  covariance  matrix  from  a  sample 
n 

of  size  n  from  a  multivariate  normal  distribution  with  nonsingular 

covariance  matrix  M.  That  is,  M  =  (l/n)E?  (Y.-Y)(Y  -  Y)',  where 

n  i=l  ~i  ~  ~  ~ 

~l*~2*.*",~n  are  *”*’,c*'  NoraaKH,  **)  • 

It  is  well-known  that  t/n(M  -M)  +  N  in  distribution,  where  vec(N) 

n 

has  a  multivariate  normal  distribution  with  zero  mean  and 

covariance  matrix  £  =  (I  +  I.  .)(M  ©  M) .  (See  Izenman  (1976)  or 

(p,p) 

Magnus  and  Neudecker  (1979).)  Choose  £  =  (I  +  I,  ..)(M  ©M  ). 

6  n  (p,p)  n  n 

It  can  then  be  verified  that  Assumptions  3.1  and  3.2  are  satisfied,  and 
so  the  results  of  this  paper  apply  to  this  example. 


(7.1) 


On  the  set  rank(PQA)  -  r,  we  have  the  representation 

T  (A)  =  n  Z  , a  U-1Trace{A’  P  aCa' X  D  (p)X'A]'1}, 
a  ysw  y  n  n  n 


where  X  is  defined  in  (6.2)  and  D  (y)  is  an  (m  *  m)  diagonal  matrix 
n  n 

a  a  2 

with  entries  / ( A ^  — y )  ,  j  =  i , i+1 , . . . , i+m-1 .  For  the  important  case 

r  =  m,  (7.1)  becomes 

(7 .2)  T  (A)  ~  n  Tr  ar  of  A  *  M  ^  A  /  a  *  v  \  1 


T  (A)  =  n  Trace{A’M_  A(x'a)  A(a'x  )"L  + 
n  n  n  n  n 


A’MA(X’A)"1Ar1(A^X  )_1  -  2A* A(X* A) (A* X  )-1}, 
“  n  n  n 

A  A  A 

where  A  is  an  (m  *  m)  diagonal  matrix  with  entries  A., A.  ,,..., A. 

n  ii+l  i+m-1 


In  particular,  for  m  =  1,  (7.2)  becomes 


(7.3) 


T  (a)  =  nCA.a'M_1a  +  ATVm  a  "  Z&'ail/ (&'&. ) 
n  ~  i~  n  ~  i  ~  n~  1 


-1 


,.A 
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Under  the  null  hypothesis,  this  statistic  is  asymptotically  equivalent 
to  the  test  statistic  given  by  Anderson  (1963).  Anderson's  statistic  is 


(7.4) 


nCX^^M  ^a  +  A.a'M  a  -  2], 
i~  n  ~  i~  n~ 


with  a  normalized  such  that  a’ a  =  1. 


II)  CANONICAL  ANALYSIS.  Another  common  use  of  eigenvectors  in 
statistics  is  in  the  canonical  analysis  of  the  joint  covariance 
matrix  for  two  random  vectors  which  are  jointly  multivariate  normal. 
For  this  car,,  K,  -  C$12C$n  ,„<!  a  -  oh,r. 


c  =  C11  C12  [Cll  C12 

C _  a  a  and  C  = 

.C21  C22j  C21  C22 


represents  the  sample  covariance  matrix  for  a  sample  of  size  n  from 
a  (p  *  q)  multivariate  normal  vector  with  nonsingular  covariance  matrix  C. 

A 

For  this  example,  T  =  C  and  T  =  C  . 

11  n  11 

By  expanding  M  in  a  Taylor  series  about  C,  we  note  that  /n(M  -  M)  -*■  N 
n  n 

in  distribution,  where  vec(N)  has  a  multivariate  normal  distribution 
with  mean  zero  and  covariance  matrix 

(7.5)  i  -  ra-M)  ®  r“V  +  t(m-m2)  ©  r_1(i-2M') 

+  I,  ,{(I-M)  ®  (M’-CM^2)  +  (M-M2)  ®  (I-M*)} 
(P.P> 

Choose  I  to  have  the  same  form  as  I  with  T  and  M  replacing  T  and 
M  respectively.  For  C ?  0,  it  can  be  verified  that  Assumptions  3.1 
and  3.2  are  satisfied,  and  so  the  results  of  this  paper  apply  to  this 
example. 


On  the  set  rank(P^A)  =  r,  we  have  the  representation 


(7.6) 


T  (A)  =■  n  Z  a  Trace{Air  P  ACa’T  X  D  (p)x’r  A]'1} 


n  u  n  n  n  n  n 
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where  Da(u)  is  an  (m  x  m)  diagonal  matrix  with  entries 
a  A  a  a  2 

(1-X.)(U>X.-2UX.)/(A  -u)  ,  j  =  i, i+1 , . . . , i+m-1 . 

J  J  J  J 

In  particular,  for  m  =  1,  (7.6)  becomes 


appendix 


In  Chis  appendix,  Che  proofs  for  section  5  are  given.  These 
proofs  are  given  in  the  sane  order  in  which  they  appear  in  section  5 
with  the  exception  that  the  proof  for  Theorem  5.1  follows  the  proof  for 
Theorem  5.2. 

Before  presenting  the  proofs,  the  following  lemmas  concerning 
quadratic  forms  involving  singular  matrices  are  needed. 

LEMMA  A.l.  Let  S  be  a  positive  serai-definite  symmetric  matrix 
of  order  (s  *  s),  x  e  range(S),  and  B  a  (s  *  k)  matrix  with  rank(B)  =  k, 
Chen  (i)  x' B(B’ SB)~B1x  is  invariant  with  respect  to  the  choice  of  the 
generalized  inverse  for  B'SB. 

(ii)  x'B(B  'SB)~B-’x  <  x'S  x,  with  equality  if  k  =  s. 

(iii)  x'S  x  S  (x'x)2(x'Sx)  1 - 

LEMMA  A  2  For  Y  ,  an  s-dimensional  random  vector,  if 
.  ~n 

(i)  Y  •*  Norma  ICO, S)  in  distribution,  with  rank(S)  =  r, 

~n 

(ii)  S  -*•  S  in  probability, 

n 

(iii)  rank( S  )  +  r  in  probability,  and 

n 

(iv)  ProbtY  c  range(S  )]  -*■  1, 

~n  n 

then  for  any  sequence  of  generalized  inverses  for  S^, 

_  2 

Y'S  Y  -*■  x  •  i-n  distribution. 

~n  n~n  * 

LEMMA  A.3  .  Let  S  and  S  be  random  matrices  such  that  S  -*•  S  in 
n  n 

distribution.  If  rank(S)  =  r  almost  surely,  and  rank(Sn)  ^  r,  then 

rank(S  )  -*■  r  in  probability.  Alternatively,  ProbCrank(S  )  *  r]  -*•  1. 
n  n 


The  proof  for  Lemma  A.l  is  straight  forward  and  can  be  found  in  Appendix 
C  of  the  author's  dissertation.  Lemma  A. 2  is  a  corrected  version  of 


Theorem  l.b  given  by  Moore  (1977).  Proof's  for  Lemmas  A. 2  and  A. 3 
can  also  be  found  in  Appendix  C  of  the  author's  dissertation. 

Finally,  the  following  special  case  of  the  theorem  given  by  Okamoto 
(1973)  is  needed  before  the  proofs  for  section  5  can  be  given. 

LEMILA  .A. 4.  If  B  is  a  (k^  x  k?)  random  matrix  such  that 
vec(B)  -  Normal(0>S)  with  rank(s)  =  k^k2>  then  rank(B)  =  min^.k^)  almost 

surely. 

— £-o£  ■The-0-r-gm  ? •  2  .  i ..  By  noting  that  I  (A)  =  l  (p  A)  is  a  sample 

o  o  o  r 

version  of  I^(A)  in  Theorem  4.2,  we  obtain 

^A-1^  rankCI^fA)]  =  (p-m)r, 

A  A 
whenever  rank(P^A)  =  r.  It  is  then  easy  to  verify  that  if  rank(P  A)  =  r 

a  A  ° 

Chen  rangeCIo(A)]  =  rangeCi  ®  (I-Pq)3.  So,  by  noting  that 

Cl®  (I-P  )>ecC(l-P  )A]  =  vecC(l-P  )A],  we  have 

(A.2)  if  rank(PQA)  =  r,  then  vecC(l-Po)A]  is  in  rangelt^A)  3 . 

The  proof  for  Theorem  5.2.i  is  completed  by  applying  Lemma  A.l.i  to 

T  (A) ,  using  B  =  I. 
n 

Proof  of  Theorem  5.2.ii.  Let  r,  =  rank(P  A),  and  let  G_  and  G_  be 
- - 1  o  11 

the  total  eigenprojections  associated  with  Che  r^  largest  eigenvalues  of 
A'P;rPoA  and  A'P^rp^A  respectively.  Also,  let  G^  =  I-G^,  and  C0  =  I-C^. 

By  noting  that  (P  AC, ) 'r(P  AG  )  =  0  and  (P  AG  )(P  AG,)'=0,  we  have 
oi  o  o  o  o  o  1 

A  A  A  A  A  A  A  A  AAA 

rank(P  A)  ^  rank(P  AG  )  +  rank(P  AG.).  Now,  since  rank(P  AG,)  =  rank(G' A' P' Tp  AC  ) 
o  oo  o  i  ol  1q  o  1^ 

A  AAA 

and  G'^A' P^fP^AG^  -*■  A' P^fP^A  in  probability,  it  follows  from  the 
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continuity  of  eigenvalues  of  symmetric  matrices  that 
a  A 

rank(P^AG^)  -*•  r^  in  probability.  So,  we  only  need  to  show  that 

A  A 

rank(?  AG  )  -*■  (r-r,)  in  probability, 
o  o  1 

To  show  this,  using  Lemma  4.1,  we  obtain  the  Taylor  series 

G  =  G  -  G  (A'PTP  A  -  A' P' TP  A)  (A1  P'  fP  A)  + 
ooooo  oo  oo 

-(A  p’TP  A)+(A'p'rP  A  -  A*p'rp  A)G  +  0(a“2). 
oo  oo  ooo  n 


A 

Using  this  expansion,  and  the  expansion  for  Pq  given  by  Lemma  4.1, 
we  obtain 

A  A 

(A.  3)  a  (P  AG  )  -*■  W  =  ~B  Z,  P.NQ.AG 

n  o  o  o  Aew  A  A  o 

in  distribution,  where  B  =  [P  -  P  aCa'p'TP  A^a'p'T]  • 

o  oo  oo  o 

Noting  that  G  and  B  are  projections  with  rank(G  )  =  (r-r. )  and 
o  o  r  J  oi 

rank(30)  =  (m-r^),  we  can  choose  matrices  and  of  dimension 

Cr  *  (r-r^)]  and  [p  *  (m-r^)]  respectively,  such  that  rank(C^)  =  (r-r^), 

rank(C  )  =  (m-r  ) ,  G  C  =  C  and  B  C  =  C  .  It  can  then  be  shown  that 
L  loll  022 

for  any  C(r-r^)  x  (m~r^)]  matrix  C, 

(A. 4)  var{[vec(C)]'vec(C‘rwC, )}  =  Cvec(TC  )]'lvec(rc  ), 

L  1  O  q 


where  Cq  -  ^Xew?\C2CC 1AC*A '  For  C  ?  °>  (A-^)  is  positive.  This  follows 
by  noting  that  £Aew^y(twU"1(u"X) V(Co+Co)PX  =  AC1CC2  *  °*  and  50 
M(Cq+C^)  f  0.  Thus,  by  Assumption  3.L.iii,  (A. 4)  is  not  zero. 

Since  (A. 4)  is  positive  for  C  +  0,  this  implies  that 


rankC varCc'TWC^ 
almost  surely, 
of  W  is  almost 


)]  =  (m-r^)  (r-r^) .  So,  by  Lemma  A. 4,  rank(CTwC^)  =  (r-r^) 
Also,  since  (r-r^)  S  rank(W)  ^  rankiC^CwC^) ,  the  rank 
surely  equal  to  (r-r^). 
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The  proof  Co  Theorem  5.2. ii  is  completed  by  noting  that 

rank(a  P  AG  )  <  (r-r  ),  and  then  applying  Lemma  A. 3. 
n  o  o  1 

Proof  of  Theorem  5.1.  This  proof  consists  of  showing  that  the  cou- 

A 

ditions  of  Lemma  A. 2  are  satisfied  when  using  a  vec[(P  -l)Al  for  Y 

n  o  ~n 

A 

and  I  (A)  for  S  .  Condition  (i)  follows  from  (A. 2).  Condition  (iv) 
o  n 

follows  from  (A. 2)  and  Theorem  5.2. ii.  If  condition  (ii)  is  satisfied, 
then  condition  (iii)  follows  from  (A.l)  and  Lemma  A. 3.  So,  it  only 
needs  to  be  shown  that  condition  (ii)  is  satisfied,  that  is,  to  show 


that  Jq(A)  -*■  Jq(A)  in  probability.  Since  -►  I  in  probability,  it 
is  sufficient  to  prove  that 


(A. 5) 


in  probability. 

Before  probing  (A. 5),  additional  notation  is  needed.  If 
A _ ,  f  A  =  ...  =  A  f  \  ,  then  define  Z. ( X  )  =  a,  u(\  )  =  a+b,  m(A  )  =  b+1 

3  i  cL  3^0  3  ■  i  3  3  ^ 

and  w(A  )  =  $  ,  . £  .}.  Also,  define  d(A,y)  =  (A-y)  and  for  X 

a  a  a+t» 

and  y  which  are  eigenvalues  of  M,  define 
u(A)  u(y)  A  A 

d(X,y)  =  Xfc) ]/[m(A)m(y) ] ,  provided  X  f  y.  By  the  continuity 

A 

of  eigenvalues,  that  is  Lemma  2.1,  we  note  that  d(A,y)  -*•  d(A,y)  in  probability. 
Also,  if  a  e  w(A)  and  0  e  w(y),  then  d(a,6)  -*■  d(A,y)  in  probability. 

So,  if  we  define  the  norm  on  all  (px  p)  matrices 
I |B| lG  =  [max  eigenvalue  of  G  ^B'GB]^,  where  G  is  a  symmetric  positive  definite 
matrix  of  order  (p*p),  then 

<A-6)  "J«S(x>!:6e0<u>'““-e,fi  ©  K  -  *a-u>C£«i(X)?i  ©  £e<0cu)Vllr» 

*  £I<(x)^M)ld(Ji’V  - 
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By  Che  arguments  of  the  previous  paragraph,  it  follows  that  the  right- 


hand  side  of  (A. 6)  goes  to  zero  in  probability.  Note  that  if  II B  Mr  0 

n  n 

in  probability,  then  ||B  1  |  P  -*•  0  in  probability,  since  T  +  T  in  probability. 

n  I  n 

So,  the  left-hand  side  of  (A. 6)  goes  to  zero  in  probability  if  the  norm 
I  I  *  I  Ip  are  replaced  by  fl'llp- 

n  A 

Thus,  since  by  (4.2)  ^aew(X)^a  ^ X  probability,  we  have 

(A-7>  Wp*»E«Sa>£ffci<..>d<“-,1,Pi  ®  PS  *  £Xe„Ep*vd<X-u)Pi  ®  PP  • 

in  probability.  The  proof  of  Theorem  5.1  is  completed  by  noting  that 
the  left  and  right  hand  sides  of  statement  (A. 7)  are  the  same  as  the  left 
and  right  hand  sides  of  statement  (A. 5)  respectively. 


Proof  of  statement  (5.3).  Let  r  =  rank(P  A),  and  let  B  be  a 

o  o 

A 

[r  x  (r-r  )]  matrix  with  rank(B)  =  r-r  and  such  that  P  AB  =  0.  By 
o  oo 

(A. 2),  we  can  apply  Lemma  A.l.ii  and  A.l.iii  to  obtain 


(A. 8) 


W  2  VV> 2 


where  b^  =  vecC  (I^-DA^B] .  As  k  -*■  <*>,  ^(A^B)  +  0  and  bfc  vec(-AB)  .which 
is  non-zero.  Thus,  the  right-hand  side  of  (A. 8)  goes  to  infinity. 

Proof  of  Theorem  5.3.i.  By  Theorem  5.2.i,  Prob[rank(P  A)  <  r]  -*■  0. 

o 

Thus,  we  only  need  to  show  that  if  H  is  false,  then 

o 

ProbC rank(P  A)  =  r,  T  (A)  >  c]  -►  1,  for  any  constant  c.  By  (A. 2),  we  can 
o  n 

apply  Lemma  A.l.iii  to  obtain 


(A.  9) 


T  (A)  S  a2(clc  )2Cc'l  (A)c  , 

n  n  ~n~n  ~n  o  ~n 


where  c  ■  vecC(P  -4)A].  Note  that  the  proof  for  I  (A)  converging  to 
~n  o  ° 

l  (A)  in  probability  given  in  the  proof  for  Theorem  5.1  does  not  depend 
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